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ABSTRACT 



This final report describes accomplishments of Project 
Discovery, a 3 -year project in Kentucky to assist teachers in creating an 
innovative learning environment for gifted and talented primary-aged 
children. Major goals focused on and achieved by the project included: (1) 

increasing the percentage of disadvantaged students identified as gifted in 
each of the 13 participating districts; (2) improving teachers’ attitudes, 
knowledge, behavior, and skills in the recognition and nurturing of 
giftedness in primary-aged children; (3) maintaining or increasing students’ 
self-esteem and increasing their creative thinking, critical thinking, and 
problem-solving skills; and (4) increasing parents’ knowledge and confidence 
in recognizing and nurturing giftedness in their children. Individual 
sections of the report provide a description of accomplishments pertaining to 
each goal and objective; demographic information; a listing of project 
products; data showing baseline and post-project profiles of gifted 
identification in participating districts; a listing of areas of giftedness 
served; a listing of professional development workshops conducted by the 
project; the project's plan of operation; a list of resources for research 
with primary-aged children; a directory of the 24 participating schools; 
summaries of interviews with Project Discovery teachers; and an analysis of 
student achievement data. (DB) 
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PREFACE 

Project Discovery, a three year initiative by the Ohio Valley Educational Cooperative 
funded by the Jacob K. Javits Gifted and Talented Students Education Act, U.S. Department 
of Education was designed to assist teachers in creating an innovative learning environment for 
primary-aged children. Discovery teachers involved students in independent investigations and 
the creation of new knowledge through original research. Children were encouraged to be 
curious and to identify problems and questions raised by their natural curiosity. This allowed 
students to build skills in problem solving, critical and creative thinking, and independent 
learning through real life experiences. The project accomplished the following goals: 

1 . To increase the percentage of disadvantaged students identified as gifted in the grade of 
formal entry into the gifted program in each district. 

2. To improve teachers’ attitudes, knowledge, behavior, and skills toward recognizing and 
nurturing giftedness in primary-aged children. 

3 To maintain or increase students’ self-esteem and increase the creative thinking, critical 
thinking and problem solving skills of those students. 

4. To increase parents’ knowledge and confidence in recognizing and nurturing giftedness 
in their children. 

Staff members of Project Discovery: 

Lois Conely Chris Luvisi Kim Sego 

Trainer Coordinator/Trainer Trainer 

For more information concerning Project Discovery contact Dr. John Rosati (502) 222-4573 
ext. 16, Ohio Valley Educational Cooperative. 
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PROJECT DISCOVERY 
FINAL REPORT 
OCTOBER 1, 1995 



Accomplishments in Carrying Out goals and Objectives 

The following is a summative discussion pertaining to each objective of Project 
DISCOVERY and the accomplishments made to date. 

GOAL 

TO NURTURE GIFTEDNESS IN YOUNG STUDENTS (PRIMARY) 

Objective #1 ASSESSMENT AND IDENTIFICATION 

By June 30, 1995, to increase the proportion of disadvantaged students identified as 
gifted in the grade of formal entry into the gifted program in each district, as 
evidenced by district demographic data. 

The project has gathered baseline demographic data of the percentage of disadvantaged 
students currently identified as gifted in the grade of formal entry into the gifted program. 

To increase the proportion of disadvantaged students identified as gifted in each district, the 
project has established an Assessment Committee. The committee has developed an 
identification instrument, Project DISCOVERY Primary Portfolio Talent Guide, which 
includes individual task assessment and portfolio assessment, relative to the six Kentucky 
Learning Goals and Learner Outcomes. The Talent Guide describes levels of identifiable 
behaviors from Beginning-Developing-Competent-to-Outstanding. The Assessment 
Committee has selected work samples from assigned performance tasks. Additionally, the 
project contracted with Dr. Donna Y. Ford-Harris, University of Virginia, National 
Research Center on Gifted Education to lead to work of the Assessment Committee. The 
Assessment Committee created a technical assistance handbook which was disseminated 
April, 1994 to assist OVEC districts in the development of strategies to implement the new 
state regulation on serving gifted students and to design procedures to increase the number 
of underserved rural and economically disadvantaged students in member districts. Finally, 
teachers in the project have received training assisting them in the identification of young 
children who are disadvantaged and gifted through observations and anecdotal evidence. 
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Presently, Dr. Joseph Petrosko, University of Louisville is conducting a statistical analysis 
of the data collected on each student in Year Three to determine the relationship between 
age, gender, socio-economic status, and achievement. Also, data has been collected to do a 
pre and post comparison on the number of students identified as gifted in grades 4-6 from 
1992 to 1995. This data will allow the project to determine its effect in increasing the 
number of economically disadvantaged students as gifted. The results of the analysis 
demonstrates that four hundred eighty-two more students were identified as gifted in 1995 
as were 1992 across the thirteen districts. In 1992, only 7% of identified gifted students in 
grades 4-6 were economically disadvantaged. In 1995, districts reported an increase of 
over 100% in the number of students from disadvantaged backgrounds identified as gifted. 
In 1995, 15.4% of all identified gifted students were economically disadvantaged. 
Furthermore, identification of ethnically diverse gifted students increased from 1.2% in 
1992 to 2.7% in 1995. Overall, OVEC school districts have 34% of all students 
economically disadvantaged and 3.6% of all students belonging to an identified ethnic 
minority group. See Appendix A for detailed charts outlining the above information. 

Objective #2 PROFESSIONAL DEVELOPMENT 

By June 30, 1995, to improve teachers' attitudes, knowledge, behavior, and skills in 
recognizing and nurturing giftedness in primary-aged students, as evidenced by 
professional development pre- and posttest comparisons, baseline and periodic teacher 
surveys, teacher interviews, and narrative reports of team progress in facilitating 
student research as captured on video and critiqued by the team. 

To accomplish objective #2, teachers participated in ten days of training during Year One to 
improve their attitude, knowledge, behavior, and skills in recognizing and nurturing 
giftedness in primary-aged students. DISCOVERY teachers, Study Team teachers, gifted 
education resource teachers, and administrators participated in a total of nine workshops 
during Year 2. During Year 3 teachers participated in eight days of professional 
development. Data has been collected at each training to measure those variables. A 
comparison of teachers’ attitude, knowledge, behavior, and skills in recognizing and 
nurturing giftedness in primary-aged students from Year 1 to Year 2 has been provided to 
OVEC superintendents. Comparisons for all three years are available in the final report. 
One of the most important concepts stressed in the teachers’ professional development is 
that their judgment about “who is gifted” should be withheld until there is evidence based 
on performance tasks rather than on “first impressions” from a child’s early ability to read 
or speak well. The project has focused on the need to teach all children problem solving 
skills, engage all children in critical thinking, and provide the materials and support so that 
all primary teachers in the project may provide the same challenges to all children in their 
classes. Teachers are instructed to look at children through the eyes of performance 



0 

ERIC 



2 



6 



assessment that is open-ended so that all children are nurtured and have an opportunity to 
demonstrate what they know and can do. This has been critical to the inclusiveness and 
success of the project to date. 

A comparison of all three years demonstrates a one-to-one correlation with time and 
learning for teachers in the project. Each year, project teachers reported higher levels of 
agreement with the thirteen evaluative questions regarding their behavior, beliefs, and 
project implementation. Appendix B details the three years of professional development 
and an comparison across the three years. 



Objective #3 INDIVIDUALIZED INSTRUCTION 

By June 30, 1995, to maintain or increase self-esteem and to increase the creative 
thinking, critical thinking, and problem solving skills of primary students served by 
the project as compared to a control group not served by the project, as evidenced by 
student portfolios evaluated through scoring rubrics and benchmarks. 

Project teachers and control group teachers were given four performance tasks to conduct 
with their primary students during the 1993-94 school year. At the conclusion of the year, 
teachers brought their scored portfolios to an assessment meeting where they traded 
portfolios with another teacher for inter-rater-reliability checks. A comparison of the two 
groups will provide evidence of the differences in relation to creative thinking, critical 
thinking, problem solving and self-esteem. Data from Year 3 is now being analyzed. 

During Year 3 teachers kept “Talent Portfolios” on all students. In the portfolio teachers 
included one piece that was original research while all other pieces reflected the each child’s 
strength(s). Trainers keep journals and record their observations of and conversations with 
all project teachers that reflect they are teaching critical and creative thinking skills and 
problem solving through real life situations to a greater degree than had been reported by 
teachers or observed by trainers prior to the beginning of the project. Additionally, 
primary teachers have reported that their students enjoy the new activities and teacher feel 
confident and excited about these experiences enhancing students' self-esteem and internal 
locus of control. Encouraging life-long learning attitudes increases the chance children will 
engage in original research. 

Appendix C is a discussion by the outside evaluator, Dr. Joseph Petrosko, University of 
Louisville in a detailed analysis of the performance tasks and portfolio data from student 
work. In brief, an analysis of the data demonstrates an increase in student performance in 
both the performance task related to original research and an overall increase in portfolio 
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scores which is a collection of “best works” by each student. See Appendix C for a 
detailed discussion of the data. 

Objective H PARENT INVOLVEMENT 

By June 30, 1995, to increase parents' knowledge and confidence in recognizing and 
nurturing giftedness in their children, as evidenced by baseline and periodic parent 
surveys and parent interviews. 

Parent involvement is critical to both the primary program and Project DISCOVERY. 
Primary teachers are encouraged to meet regularly with parents to discuss the strengths of 
their students and to help identify the interests of each child. Two parents are represented 
on both the Assessment Committee and Advisory council and assisted in the writing of the 
parent brochure. The brochure, “Discovering the Talents of Your Child,” has been 
developed, field tested with parents, and disseminated to all schools in Project 
DISCOVERY. 

To construct baseline data, a survey of parents whose children are members of 
DISCOVERY classrooms will be conducted during September, 1994, to measure parents' 
knowledge and confidence in recognizing and nurturing giftedness in their children. 
Workshops were conducted at all 16 school this fall to increase parents' knowledge and 
confidence in recognizing and nurturing giftedness in their children. 

Each school collected survey data from parent workshops conducted in Years 2 and 3. 
Parents reported an increase in their understanding of giftedness, an increased awareness of 
how to nurture giftedness in their own child, and an increase in communication with their 
child’s teacher. Each parent received a copy of Discover the Talents of Your Child 
produced by Project Discovery. In most cases, teachers reported using the brochure to 
begin a discussion with each child’s parent(s) at the beginning of the school year. A copy 
of the Discover the Talents of Your Child has been provided in previous reports. 

DEMOGRAPHIC INFORMATION 

Project DISCOVERY participants (students) represent an age range of five years to nine 
years old. One hundred ten primary classrooms from all thirteen OVEC districts 
participated during Year 3. Of those students in DISCOVERY classrooms, 34% are 
economically disadvantaged (1,024), while 3% are from racial groups other than Caucasian. 
Three percent of DISCOVERY students have one or more handicapping conditions. The 
2,601 students who participated in the project are demographically representative of the 
total districts’ population. 
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PROJECT DISCOVERY 1994-95 DEMOGRAPHIC PROFILE 
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Anchorage 


36 


0 


36 








Bullitt 


Brooks 


67 


19 


67 










Maryville 


82 


40 


82 
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18 










Old Mill 


48 


8 


48 
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PRODUCTS CREATED BY 
PROJECT DISCOVERY 



Project Discovery Talent Guide 

Project Discovery Performance Task Guide 

Discover The Talents of Your Child: A Parent Brochure 

Real Life Discoveries : A Collection of Student-Directed Research by 
Primary Aged Children 

A Report: “ Gifted Disadvantaged and Minority Students: An Issue of 
Equity and Excellence” 

A Report: “Kentucky’s Primary School Program: Teacher Self-Report 
of Implementation of the Critical Attributes Relative to Mix-aged 
Class Arrangements and Poverty Level” 

Project Discovery Technical Assistance Handbook: Discovering the 
Talents of Rural and/or Economically Disadvantaged Gifted Students 

Year One Evaluation Report 

Year Two Evaluation Report 

Final Report 
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C=Caucasian, A=African American, A-P= Asian Pacific, H=Hispanic, 0=0ther 



STUDENTS SERVED BY OVEC DISTRICTS IN 1994-95 
BY AREA OF GIFTEDNESS 



SCHOOL 

DISTRICT 


GRADE 

LEVEL 


NUMBER OF STUDENTS SERVED 






Intellectual 


Academic 


Creativity 


Leadership 


Performing 

Arts 


Anchorage 


4-6 


15 


36 


17 


10 


2 


Bullitt 


4-6 


154 


63 


27 


47 


35 


Carroll 


4-6 


0 


36 


3 


2 


0 


Eminence 


4-6 


27 


27 


27 


27 


0 


Gallatin 


4-6 


28 


12 


24 


13 


11 


Grant 


4-6 


66 


17 


12 


17 


12 


Henry 


4-6 


49 


16 


12 


0 


0 


Oldham 


4-6 


263 


368 


199 


198 


217 


Owen 


4-6 


36 


36 


36 


24 


0 


Shelby 


4-6 


77 


77 


65 


5 


13 


Spencer 


4-6 


8 


25 


9 


11 


15 


Trimble 


4-6 


13 


13 


8 


15 


0 


West Point 


4-6 


11 


11 


11 


11 


11 


TOTALS 




747 


737 


450 


380 


316 
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PROJECT DISCOVERY 



YEAR 3 Professional Development Workshops 



Administrators: 


1994-95 OVEC PRIMARY ROUNDTABLE SCHEDULE 


September 15, 1994 
November 17, 1994 
February 16, 1995 


Administrators are kept up-to-date 
through meetings of OVEC Primary 
Roundtable 



April 20, 1995 

All Roundtable meetings will be held at the Oldham County Board Offices and begin at 1 :PM. 
DISCOVERY Teams and Gifted Education Resource Teachers: 



August 1-2, 1994 


Facilitating Independent Learning in the Primary Classroom, 
Dr. George Betts, University of Northern Colorado 


August 3-5, 1994 


Three days of AIMS (Activities Integrating Math and Science), 
ACES (Activity Centered Elementary Science) and Project 
WILD, DISCOVERY Staff 


October 8, 1994 


Using the DISCOVERY TALENT GUIDE to Recognize and 
Nurture the Strengths of All Children and Differentiating the 
Curriculum for Talented Learners through Learning Centers 
using a Multiple Intelligences Approach and Grouping and 
Regrouping Strategies, DISCOVERY Staff 


November 19, 1994 


Teaching Young Children to Be Investigators, Dr. Alane Starko, 
Eastern Michigan University 


April 29, 1995 


Inter-rater Reliability Scoring for Project DISCOVERY Talent 
Guide (8-1 LAM) 



All meetings will begin at 8:30 and conclude by 2:30 unless otherwise noted and be held at 
Camden Station Elementary in Crestwood. 
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Administrators : 



Project DISCOVERY 

Year 2 Professional Development Workshops 



September 16, 1993 Administrators are kept up-to-date through meetings of 

November 11, 1993 OVEC Primary Roundtable. 

January 20, 1994 
March 17, 1994 



Administrators and Gifted Education Coordinators; 

February 18, 1994 Identifying Rural Disadvantaged Students, Dr. H. Spicker 

Study Teams. Sparkers and Gifted Resource Teachers; 

October 19, 1993 Orientation for Study Teams, DISCOVERY staff 

February 12, 1994 Introduction to Used Numbers . Dr. Charles Thompson 



DISCOVERY Teams and Gifted Resource Teachers: 



October 7, 1993 



October 16, 1993 



November 13, 1993 



February 12, 1994 



Nurturing Giftedness Through and Integrated Math/Science 
Approach, DISCOVERY Staff 

Designing Invitations for Young Children to Conduct 
Original Investigations, Dr. Alane Starko 

Extending Used Numbers to Encourage Original Research, 
Dr. Charles Thompson 

Follow-up to Designing Invitations for Young Children to 
Conduct Original Investigations, Dr. Alane Starko 



DISCOVERY and Control Group Teachers: 

March 12, 1994 Follow-up to Nurturing Giftedness Through an Integrated 

Math/Science Approach Through Performance Assessment, 
DISCOVERY Staff 



June 4, 1994 (8:30-ll:AM) Using the DISCOVERY Talent Guide to Nurture and 

Identify the Strengths of Primary Students, Ken Jones 



Note: 

O 

ERIC 



♦Saturday trainings will be held at South Oldham High School in Crestwood. All other 
trainings will be held at Oldham County Board of Education in Buckner. 
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Project DISCOVERY 



Professional Development Workshops 
Year 1 



Administrators: 

November 13 1 Day 



Orientation for Administrators, 
Dr. Gina Schack 



Tparhsrs of Gifted and Talented: 
November 23 1 Day 



Being an Effective Gifted Resource 
Person," Dr. Deborah Burns 



Teachers of Gifted and Talented and Primary Teachers; 



November 14 


1 Day 


H pprngni23iig Giftedness Among 
Disadvantaged Populations," 
Dr. Gina Schack 


December 5 


1 Day 


"Making Curriculum and Instruction 
More Open Ended," Dr. Gina Schack 


January 9 


1 Day 


"Introduction to The Tfced Number 
Series: Counting. Sorting. Measuring*" 
Dr. Charles Thompson 


January 16 


172 Day 


"Introduction to Portfolio Assessment, 
Using a Rubric to Score the Portfolio," 
Ken Jones 


February 1 


1 Day 


"Looking for Data in All the Right 
Places: Conducting Original Research 
with Young Investigators," 

Dr. Gina Schack 


February 12 


1 Day 


Continuation of February 1 Session, 
Dr. Gina Schack 


March 20 


1 Day 


"PVplnring The Used Number Series*" 
Dr. Charles Thompson 



O 
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May 8 



1/2 Day 



July 21-23 



3 Days 



Control Group Teachers: 



January 16 1/2 Day 



May 8 



1/2 Day 



"Portfolio Assessment: 
Collaborative Auditing of the 
Scoring Process," Ken Jones 

"Discovering Science to Nurture 
Giftedness in Young Children," 
Kentucky Department of Fish and 
Wildlife and Kentucky Science and 
Technology Council 



"Introduction to Portfolio Assessment, 
Using a Rubric to Score the Portfolio," 
Ken Jones 

"Portfolio Assessment: Collaborative 
Auditing of the Scoring Process," 

Ken Jones 
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SUMMARY OF PARTICIPANT RESPONSES TO SURVEY FOR YEARS 

ONE, TWO, and THREE 



QUESTION 


YEAR ONE 


YEAR TWO 


YEAR THREE 


The workshop outcomes were clear. 


4.12 


4.23 


4.47 


The workshop outcomes were met. 


4.06 


4.17 


4.4 


The activities were relevant. 


4.09 


4.21 


4.37 


The presenter was organized, informed 
and supportive. 


4.38 


4.4 


4.45 


My concerns were addressed. 


3.9 


4.03 


4.18 


I understand the objectives of Project 
Discovery. 


4.15 


4.32 


4.34 


I understand the broadened approach to 
gifted education. 


4.22 


4.32 


4.34 


I understand how this project fits into the 
primary. 


4.22 


4.3 


4.43 


I understand the instructional approach 
that uses investigations. 


3.96 


4.15 


4.39 


I know how to identify and instruct a 
diverse population of gifted learners in 
my classroom. 


3.65 


3.96 


4.13 


I believe I can nurture giftedness and 
cause it to emerge. 


4.02 


4.17 


4.25 


My teaching behavior nurtures 
giftedness. 


3.91 


4.12 


4.19 


I utilize a variety of strategies and 
approaches to meet the needs of gifted 
students in my classroom. 


4.02 


4.17 


4.3 



1=DIS AGREE, 5=AGREE 
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RESOURCES FOR RESEARCH 



Looking for Data in All the Right Places 
Alane Starko and Gina Schack 
Creative Learning Press 
P.O. Box 320 

Mansfield Center, CT 06250 

Activities Integrating Math and Science (AIMS) 
AIMS Education Foundation 
P.O. Box 8120 
Fresno, CA 93747 
1-209-255-4094 

Used Numbers: Real Data in the Classroom 
Dale Seymour Publications 
P.O. Box 10999 
Paloalto, CA 94303 

Delta Science Modules 
P.O. Box 915 
Hudson, NH 03051 

Activity-Centered Elementary Science (ACES) 
167 W. Main Street 
Room 904 

Lexington, KY 40507 
(606)233-3502 

Creative Problem Solving for Kids 
Scare Yourself to Sleep . Rose Impey 
Mv Great Aunt Arizona 




21 



33 



PROJECT DISCOVERY SCHOOLS 
(Alphabetical by District) 



ANCHORAGE PUBLIC SCHOOLS 

Anchorage Elementary School 
11400 Ridge Road 
Anchorage, KY 40223 
(502)245-2121 

BULLITT COUNTY PUBLIC SCHOOLS 

Brooks Elementary 
1800 East Blue Lick Road 
Shepherdsville, KY 40165 
(502)957-4795 

Maryville Elementary 
4505 Summers Drive 
Louisville, KY 40229 
(502)955-6553 

Nichols Elementary 
10665 Highway 44W 
West Point, KY 40177 
(502)922-4718 

Old Mill Elementary 
11540 Highway 44E 
Mt. Washington, KY 40047 
(502) 955-7696 

Overdale Elementary 
651 Overdale Drive 
Louisville, KY 40229 
(502)957-2160 

Roby Elementary 
1148 Highway 44E 
Shepherdsville, KY 40165 
(502)955-9200 
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CARROLL COUNTY PUBLIC SCHOOLS 



Kathryn Winn Elementary 
9th & Hawkins Streets 
Carrollton, KY 41008 
(502)732-7090 

EMINENCE INDEPENDENT SCHOOLS 

Eminence Elementary 
200 W. Broadway 
Eminence, KY 40019 
(502)845-5428 

GALLATIN COUNTY PUBLIC SCHOOLS 

Gallatin County Elementary 
P.O. Box 148 
Warsaw, KY 41095 
(606)567-4723 

GRANT COUNTY PUBLIC SCHOOLS 

Crittenden-Mt. Zion Elementary 
R.R. 2 

Dry Ridge, KY 41035 
(606)428-2171 

Dry Ridge Elementary 
275 School Road 
Dry Ridge, KY 41035 
(606)824-4484 

Mason Corinth Elementary 
Rt. 1 Heekin Road 
Williamstown, KY 41097 

HENRY COUNTY PUBLIC SCHOOLS 

Campbellsburg Elementary 
P.O. Box 280 

Campbellsburg, KY 40011 
(502)532-7346 
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OLDHAM COUNTY PUBLIC SCHOOLS 



Crestwood Elementary 
6500 W. Highway 146 
Crestwood, KY 40014 
(502)241-8401 

Goshen Elementary 
P.O. Box 116 
Goshen, KY 40026 
(502)228-0101 

LaGrange Elementary 
500 West Jefferson Street 
LaGrange, KY 40031 
(502)222-9454 

OWEN COUNTY PUBLIC SCHOOLS 

Owen County Primary 
1925 Highway 22E 
Owenton, KY 40359 
(502)484-5439 

Owen County Elementary 
1945 Highway 22E 
Owenton, KY 40359 
(502)484-3417 

SHELBY COUNTY PUBLIC SCHOOLS 

Southside Elementary 
800 Eighth Street 
Shelbyville, KY 40065 
(502)633-3452 

SPENCER COUNTY PUBLIC SCHOOLS 

Spencer County Elementary 
P.O. Box 542 
Taylorsville, KY 40071 
(502)477-3260 
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ST. FRANCIS SCHOOL 



St. Francis School 
Highway 42 
Goshen, KY 40026 
(502)228-1197 

TRIMBLE COUNTY PUBLIC SCHOOLS 

Milton Elementary 
Route 2 Box 75 
Milton, KY 40045 
(502)268-3322 

WEST POINT INDEPENDENT SCHOOLS 

West Point Elementary 
P.O. Box 367 
West Point, KY 40177 
(502)922-4797 
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Report of Interviews with Teachers Participating in Project Discovery 
A Project of the Ohio Valley Educational Cooperative (OVEC) 



Joseph M. Petrosko 
School of Education 
University of Louisville 
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SUMMARY 



In February 1994, 13 teachers participating in Project Discovery were 

interviewed to determine their attitudes and opinions about the project# 
Teachers represented eight separate school districts (or schools)# 

Most teachers agreed or strongly agreed that they have an understanding of 
how to recognize and nurture giftedness in primary aged students. Furthermore, 
most teachers agreed that they could instruct gifted learners in their 
classrooms# Ten of 13 teachers interviewed strongly agreed with the statement 
that they used Project Discovery methods and materials in their classrooms . 

A majority of teachers (8 of 13) rated workshops and high or very high in 
quality. Most suggestions for improvement of workshops dealt with increasing 
the "hand-on" experiences available to workshop participants. Teachers 
expressed a great deal of satisfaction with field visits by Kim, Lois, and 
Chris (12 of 13 respondents rated visits high or very high in quality). 
Suggestions for improvement dealt mostly with increasing communication 
regarding visits and improving the fit between activities brought by visitors 
and ongoing classroom activities# 

Regarding future training, several teachers expressed interest in more 
training dealing with ACES and AIMS. There was very strong endorsement for 
training on how teachers can manage (i.e., facilitate) independent 
investigations by students# 
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Thi. s report describes a study of teachers parti, cipating in Project 
Discovery, operated by the Ohio Valley Educational Cooperative (OVEC). 
Teachers were interviewed to obtain their attitudes and opinions about 
participation in the project. 

Me thod 

Teachers were individually interviewed and were told that information 
collected would be reported in such a way that no respondent could be 
identified. Questions consisted of scaled items requiring quantitative 
judgments and open-ended items in which subjects could respond in their own 
words. A copy of the three-page interview guide appears in Appendix A. 

Data were collected in cooperation with the project director, who, in 
collaboration with the author of this report, selected teachers that would be 
representative of all those participating in the project. Interviews occurred 
on February 12, 1994, a Saturday when workshop training was being offered. 

Two workshop experiences took place at a high school located in a 
participating district. Interviewees left the workshop they were attending and 
went to the teacher’s lounge of the school for the interviews, which lasted 10 
to 20 minutes. A total of 13 persons were interviewed. They came from eight 
separate school districts (or schools). 

Results 

Most of the persons interviewed had considerable teaching experience. 
Average number of years experience was 14.4 years. 

Teacher abilities and behaviors 

Teachers were asked to rate each of four statements on a five step scale 
with 1 = strongly disagree, 2 = disagree, 3 = undecided, 4 = agree, and 
3 = strongly agree. Shown below is each statement and the mean response of the 
13 respondents. 
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You have 
s tuden t s • 


an understanding of how 


to recognize 


giftedness in primary-aged 


Strongly 

disagree 

1 


Disagree 


Undecided 


Agree 


Strong ly 
Agree 








Mean - 4.15 




You have 
s tudent s . 


an understanding of 


how to nurture gif redness in 


pr ima ry-aged 


Strongly 
d isagree 
1 


Disagree 


Undecided 


Agree 

4 


Strongly 
Agree 
5 








Mean = 3.92 




You know 


how to instruct gifted 


learners in your classroom. 




Strong ly 
d isagree 
1 


Disagree 

2 


Undec ided 
3 


Agree 

4 


Strong ly 
Agree 
5 








Mean = 4.00 




You have 
ins true t 


been using some of what you have 
gifted learners in your classroom. 


learned in Project 


Discovery to 


Strongly 

disagree 

1 


Disagree 


Undecided 


Agree 


Strong ly 
Agree 



Mean = 4.69 



Elaboration of ratings Following the four scaled items, respondents were 
invited to make comments in their own words: "If you wish, you may explain or 
elaborate upon any of your answers." Some respondents made general comments. 
Often these were a follow-up on the previous scaled item they had answered, in 
which they had been asked whether they were using Project Discovery materials 
in their classrooms. 

All interviewees made at least a general statement that they were using 
project materials in some way, or were in some way engaging in teaching 
practices that reflected the project. Spontaneously, some teachers mentioned 
specific material. These were the number of times that particular items were 
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mentioned: AIMS, 9; Looking for Data in All the Right Places, 8; ACES, 7; 

Used Numbers, 6. Below are paraphrases of comments made by teachers in 

response to the item. 

(Note. In the following summary, and all subsequent person-by-person 
summaries, the number used to identify a teacher is an arbitrarily chosen 
identifier.) 



Teacher 



12 



Comments 



13 



- Project materials fit the curriculum framework being used 
in the district and state. 

- Have shared materials with non-project teachers 

- Have changed mind about children's level of ability; believe 
that children are capable of independent thinking. 

- Have shared materials with non-project teachers. 

- Have shared materials with teachers of children 
at other grade levels. 

- Have found that activities promoted in the project, like 
interviewing, can work with primary age children. 

- Project helped confirm pre-existing belief in "underdog" 
children who are often ignored. 

- Project reaffirmed belief that gifted children need to have 
more challenging work, not just more work. 

- Project made me realize that gifted children aren't perfect. 

- Have used methods described in the project more than materials. 
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Workshops 

Teachers were asked to rate workshops on a five step scale of quality with 
1 = very low quality, 2 = low quality, 3 = medium quality, 4 — high quality, 

and 3 = very high quality. Shown below is the mean response of the 13 

respondents • 



Level of quality of training: the workshops 



Very low 
quality 
1 



Low 

qua 1 i ty 



Med ium 
quality 



High 
qua 1 i ty 

4 



Very high 
qua 1 i t y 
5 



Mean = 3.69 

Elaboration on the workshop rating Of the 13 teachers, 10 made explicit 
comments of some kind that were a follow-up to their quality ratings. Of these 
10 teachers, 7 were generally positive about one or more workshop experiences. 
Of the 10, two teachers felt that workshops did not require activities on the 
part of participants, and two felt that topics were repetitive. Comments are 
paraphrased below. 



Teacher 

1 

2 

3 

4 

5 

6 

8 

10 

11 



Comments 



- Workshop materials allow me to do things with students. 

- Workshops have been overly philosophical and not as action- 
oriented as I would like. 

- Liked Used Numbers, AIMS, Elaine Starko workshop, and ACES. 

- Used Numbers workshop and Project Wild were good. 

- Too much lecturing and some topics repeated too much. 

- Quality is good, but this year's topics are a repeat of last 
year 1 s . 

- Feel talked at rather than participating. 

- Liked Used Numbers and AIMS and ACES training. 

- There were rough spots in the early workshops, but these were 
fixed. Things have improved. 



12 



- Liked Elaine Starko workshop, 
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Opinions about improving the workshops Of the 13 teachers, 11 
or suggestions related to improvement of workshops. Five teachers 
to the effect that they preferred activities rather than lectures 
Comments are paraphrased below. 

Teacher Comments 



made comments 
made comments 
in workshops. 



1 - Instead of just lecturing during workshops, teachers need 

time to prepare rubrics. Some individual mentoring on this 
would be helpful. Would suggest that teachers get a stipend 
for preparation time, not just for coming to a workshop. 

2 - Hands-on work is good. Actually doing something myself (e.g., 

like in Used Numbers) is good for me. 

3 - Rubrics don't work for very young (K-l) children. They are 

not even at the emergent level. 

4 - Some presenters talk down to teachers. The materials are good, 

but the presentation is sometimes lacking. 

3 - Try to stress things that are actually used in the classroom. 

6 - Hands-on work is very useful. 

8 - The more active the learning, the more I benefit. 

9 - The workshops have been very helpful in suggesting techniques 

for teaching whole classes of children. However, the workshops 
have not been focusing on f inding/ identify ing the gifted or 
talented child. 

10 - The less successful ones were lectures in which there was 

no active involvement. 

12 - Sometimes fellow workshop participants (the other teachers) 

have not been very well behaved. Too much irrelevant talking. 

13 - Don't like workshops during the school week. 
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Visits by Kim, Lois, and Chris 

Teachers were asked to rate visits to classrooms by Kim, Lois, and Chris on 

a five step scale of quality with 1 = very low quality, 2 = low quality, 

3 = medium quality, 4 = high quality, and 5 = very high quality* Shown below 

is the mean response of the 13 respondents. 

Level of quality of training: visits by Kim, Lois, Chris 

Very low Low Medium High Very high 

quality quality quality quality quality 

1 2 3 4 5 

Mean = 4.75 



Elaboration on the visits rating Of the 13 teachers, six made explicit 
comments of some kind that were a follow-up to their quality ratings. Of these 
six teachers, five made generally positive comments about the field visitors. 
Comments are paraphrased below. 



Teacher 



2 

3 

4 

7 

8 



Comments 



- Biggest issue is creative writing: getting kids motivated to 

write, so that they will do well on the KIRIS exams. 

- Positive personalities help the program participants. 

- Very good information obtained; very good communica t ion-“i t is 
peer to peer, not top down. 

- Extremely helpful to everyone—very available and adaptable. 

- Kim and Lois have visited me and been very helpful. 

- Visits are strongest point of the project. 



Opinions about things to improve visits Of the 13 teachers, eight made 
positive comments about the visits; comments indicating satisfaction with the 
services received. Some teachers had specific suggestions, which are 
paraphrased below. 
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Teache r 



Comments 



2 


- For planning purposes, would like to have information on what 
visitors will be doing. 


5 


- It would help to increase communication about what teachers 
are doing and how to fit this into what the visitors will be 
doing • 


6 


- Increase feedback and increase the turnaround time on questions 
and information needed by teachers . 


10 


- Would be good to have visitors interact with smaller groups of 
children, rather than the whole class, 

- Need help on how to pose good questions as a follow-up to 
classroom activities. 


13 


- Try to bring activities when there is a visit. That helps a 
lo t • 



Future training 

Respondents were asked the following open-ended question: "Generally 

speaking, what things would be most helpful to you in terms of future 



training?" 


Responses are summarized below. 


Teacher 


Comments 


i 


- More ACES — how to use ACES, 


3 


- As someone who was a kindergarten teachers and now teaches 
K-l, I have to learn how to deal with grade 1, 


4 


- The next step is focusing on how to completely implement 
Project Discovery activities in the classroom. For example, 
how to teach kids good questioning skills. 


6 


- Providing mentorship for teachers would be he lp f ul--e • g. , 

looking at teacher materials and activities to see if it’s on 
track • 


7 

8 


- Understanding of rubrics and how to judge material and 
activities in the teacher's own classroom. 

- AIMS and ACES training. Teachers who used to teach grade 4 
need to know the expectations of primary teachers (now that 
grade 3 is a combined grade 3-4 within primary). 



8 



8 



9 - Would be useful to learn more about tests to identify the 

disadvantaged gifted (e.g., Raven's). This might include a 
day of training on using the tool. 

10 “ More things like AIMS and ACES. It would be good to have 

better integration of Project Discovery with material coming 
from the state. Project Discovery has put teachers one step 
ahead of KERA. 

11 - Increase the amount of Project Discovery activities directed to 

the Language Arts. 

12 - Increase science training. 

13 - Portfolio tasks — clarify expectations of portfolio tasks. 



Possibilities for future training Teachers were asked their opinion of two 

possible training opportunities. Each is shown below, along with the mean 

score it obtained. The items were rated on a five step priority scale with 

1 = very low priority, 2 - low priority, 3 = medium priority, 

4 = high priority, and 3 = very high priority. 

Continuing training in: posing questions for 

original investigations (with Elaine Starko) 

Very low Low Medium High Very high 

priority priority priority priority priority 

1 2 3 4 3 

Mean = 3*38 



Training on how teachers can manage (i.e., facilitate) 
independent investigations by students 



Very low 
priority 
1 



Low 

priori ty 

. ..2 



Medium 
prior i ty 

...3 



High 

priority 

4 

Mean = 4.23 



Very high 
priority 
5 



Cone lus ions 

Data from this interview study revealed high levels of teacher satisfaction 
with Project Discovery workshops and very high levels of satisfaction with 
field visits by project staff. Teachers especially liked workshop experiences 
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with "hands-on” activities and components that could be applied to their own 
classrooms. They also liked it when Project Discovery staff who visited them 
brought materials and activities that could be fit into ongoing classroom 
activities. There was strong endorsement of future training that would help 
teachers facilitate independent investigations by students. 
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Interview Guide 
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Project Discovery 
Teacher Interview Form 



J. M. Petrosko 
University of Louisville 



1. Date of interview: 

2. How many years of teaching experience do you have?: 

3. Do you have special background or training in any educational area, 
if so, what is it?: 

4. Before Project Discovery began, did you have an interest in gifted/talented 
education? If so, how did the interest arise? 



3. I am going make a series of statements. After each I will ask your 
opinion of the statement: whether you strongly disagree, disagree, 

are undecided, agree, or strongly agree with it. 



SD D U A SA 



You have an understanding of how to recognize giftedness 
in primary-aged students. 



1 2 3 4 5 



b. You have an understanding of how to nurture giftedness 
in primary-aged students. 



1 2 3 4 5 



c. You know how to instruct gifted learners in your classroom. 12345 



d. You have been using some of what you have learned in 12345 

Project Discovery to instruct gifted learners in 
your classroom. 



e. If you wish, you may explain or elaborate upon any of your answers 
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Project Discovery interview 2 

I would like you to rate several things on a 5-step quality rating scale. 
l=very low qual. 2=low qual. 3=medium qual. 4=high qual. 5=very high qual. 



6. Level of quality of training: the workshops 12345 

a. Why did you rate it that way? 



b. What specific things would you change to improve the workshops? 



7. Level of quality of training: visits by Kim, Lois, Chris 1234 

a. Why did you rate it that way? 



b. What specific things would you change to improve the visits? 
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Project Discovery interview 3 

I would like ask your opinion of future training that could be offered as 
part of Project Discovery. 

8 • Generally speaking, what things would be most helpful to you in terms of 
future training? 



Below are possibilities for future training. 

9. Rate each on how high a priority this would be for you in terms 
of your own interests and needs* 

1 -very low priority 2 =low priority 3 =medium priority 4 =high priority 
3 =very high priority 

a. Continuing training in: posing questions for original 12343 

investigations (with Elaine Starko) 

b. Training on how teachers can manage independent 12343 

investigations by students 



Thank you for participating in this interview. 



o 
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Comparison of Project Discovery Students From Year 2 and Year 3 of the Program: 



Ratings of Performance Tasks and Portfolios 



Dr. Joseph M. Petrosko 
School of Education 
University of Louisville 



Ohio Valley Educational Cooperative (OVEC) 
August 1995 
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Executive Summary 



1 



Comparison of Project Discovery Students From Year 2 and Year 3 of the Program: 
Ratings of Performance Tasks and Portfolios 



This report gives results of a study comparing data from 1299 children who had experienced 
Project Discovery in its second year (Year 2) with data from 2553 children who had experienced the 
program through its third year (Year 3). Most children in the study (95%) were White Non-Hispanic 
and were from five to eight years of age. Children were compared on two sets of teacher ratings: 
(a) a set of ratings of a performance task and, (b) a set of ratings of an overall portfolio. 

For both performance tasks and portfolios, data consisted of 16 learner outcome ratings thati 
were related to six learning goals derived from Kentucky’s 1990 statewide education reform law. For 
example, learner outcome 1.0 was Finding and Gathering information and Ideas. This was related 
to Kentucky Learning Goal 1, Applying Basic Skills. Each of the 16 outcomes (designated from 1.0 
to 6.3) was rated on a five-step scale, with scale points indicating increasingly greater levels of 
accomplishment: 0 = Not observed, 1 = Beginning, 2 = Developing, 3 = Competent, 4 = Outstanding. 

For the performance task, children in the Year 3 group had significantly higher ratings than 
children in the Year 2 group on 11 of 16 learner outcomes. For the entire portfolio, Year 3 group 
children had significantly higher ratings on 15 of 16 learner outcomes. Consistent results were 
obtained with both univariate and multivariate statistical tests (independent t tests and multivariate 
analysis of variance). 

Year 3 data were analyzed to determine the effects of several background variables on 
student outcomes. Significant differences were found between two ethnic groups: in some 
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outcome areas, White Non-Hispanic students had higher means than African-American students. 
However, there were comparatively few students in the latter category. 

Analysis revealed that females had higher scores than males. Furthermore, there were 
significant differences among children from the lowest economic group (eligible for free lunch), the 
middle economic group (eligible for reduced cost lunch), and the highest group (not eligible for free 
or reduced lunch). The higher the economic level, the higher the mean score that was obtained. 

Additional research is necessary to more completely describe and more fully explain the 
effects of ethnicity, economic level, and other variables on the ratings of student work. 
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The purpose of this report is to describe results of Project Discovery on student outcomes. 
Children in Year 2 of the program and in Year 3 were given a common performance task and also 
completed a portfolio of tasks related to research and problem solving. Tasks and portfolios were 
rated on 16 learner outcomes related to the learning goals adopted in the state as part of the Kentucky 
Education Reform Act (KERA) of 1990. 

Results 

Frequency distributions on background variables 

Students were measured on several variables related to their personal background or status 
in the program. Table 1 shows these distributions. 

As can be seen in Table 1, over 75% of students were ages 6, 7, or 8, and they were evenlyi 
spread out among those ages. Regarding economic level, most children (64%) were rated high, with 
8% rated medium and 28% rated low. These designations were categories that were chosen to 
characterize a child's status in terms of the federally supported school lunch program. They are 
relative categories— low was lower than medium, and medium was lower than high— but low did not 
mean extreme poverty and high did not mean extreme affluence. Low meant the child was eligible 
for free lunch, medium meant eligibility for reduced-cost lunch, and high meant no eligibility for free 
or discount lunch. 

Also shown in Table 1 are data on ethnicity— almost all children (95%) were White Non- 
Hispanic or African-American (4%). Gender was almost evenly divided between boys and girls. 
Finally, the variable Group referred to the distinction between what percentage of the data came from 
children getting Project Discovery activities during Year 2 (37%) and the percentage coming 
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from children during Year 3 (62%). 

In the statistical analyses that follow, some data were removed from the Year 2 data set. Data 
from subjects who were designated Control (i.e., not participating in Project Discovery) were not 
analyzed. Thus, the maximum number of subjects for the analyses that follow were: 1299 for Year 
2 and 2554 for Year 3. 
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Table l 

Frequency Distributions on Five Background Variables for Project 
Discovery Students: Age, Economic Level, Ethnicity, Gender, Group 



Variable: Age 



Value 



Frequency Percent 



9 

8 

7 

6 

5 

Missing 



269 


7 


1135 


28 


1057 


26 


928 


23 


643 


16 





2 



4096 100 



Variable: Economic 
Level 



Value 

High 

Medium 

Low 

Missing 



Frequency Percent 



2625 

307 

1127 

22 - 

4096 



64 

8 

28 i 
1 



100 



Variable: Ethnicity 



Value 



Frequency Percent 



African-American 


148 


White Non-Hispanic 


3875 


Other 


37 


Missing 


if. 




4096 



4 

95 

1 

1 

100 



Variable: Gender 



Value 



Frequency Percent 



Female 

Male 

Missing 



1946 48 

2112 52 

2S_ I 

4096 100 



Variable: Year 



Value’ 



Frequency Percent 



Year 2 
Year 3 
Missing 



1505 37 

2554 62 

22 _ 1 

4096 100 
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Differences between the Year 2 and Year 3 groups 

Data on performance tasks Table 2 shows mean scores on the performance tasks for Year 2 
and Year 3 subjects, and it also shows results of independent t-tests used to compare the means. If 
the equality of variances assumption of the independent t statistic was met, the t with the standard 
number of degrees of freedom (df) (i.e., n, + n 2 -2) was reported. However, if the assumption was 
not met, the t with conservative df was reported. This explains why there were large differences 
among the df values, for example, df = 2658 for the test involving goal 1.1, but df = 1915 for the test 
of goal 2.0. 

The analysis revealed that, using the significance criterion of g < .05, there were significant 
differences in favor of the Year 3 group for 1 1 of the 16 assessment criteria: criteria numbers 1.1} 
1.2, 1.3, 2.0, 3.1, 4.3, 5.1‘ 5.2, 6.1, 6.2, and 6.3. On two areas, criteria 3.2 and 3.3, mean scores 
were higher for Year 2 than Year 3. 

In addition to the univariate tests reported above, a multivariate analysis of variance 
(MANOVA) was performed, with the 16 performance task ratings as dependent variables and the 
Year 2 vs. Year 3 designation as the independent variable. Data from fewer subjects were analyzed 
because it was necessary to have ai complete data set for each subject (i.e., no missing data on any 
of the 16 dependent variables and the single independent variable). Nevertheless, the data set was 
substantial, with the n for the MANOVA consisting of 2307 cases. The multivariate Pillai's 
trace criterion of .06015 was converted to a statistically significant F(16, 2290) = 9.16, p < .0005. 
Thus, multivariate results corroborated the differences between the Year 2 and Year 3 groups, found 
in the results of the separate t-tests. 
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In addition to the data on the 16 separate ratings for the performance tasks, other data were 
analyzed. New variables were created that were averages over subsets of the 16 rating scale 
categories; the separate ratings within each goal were averaged to yield six goal rating items. Table 
3 shows means and t-test comparisons for the 6 goal areas. There was considerable consistency 
between these findings and those of the separate areas shown in Table 6. Three goal areas were 
statistically significant in favor of the Year 3 group: goal areas 1, 2, and 6. One goal area, number 
3, favored the Year 2 group. Goal areas 4 and 5 showed no statistically significant difference (at g 

< .05) between the two groups. A MANOVA on the six average goal areas yielded a multivariate 
Pillai's trace criterion of .04213 which converted to a statistically significant F(6, 2383) = 17.47, g 

< .0005. 
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Table 2 





Mean Scores and Independent 


£.-Tests 


Comparing 


Year 2 


and Year 


3 




Subjects for Performance 


Tasks Rated on 16 


Learner 


Outcomes 


Goal 


Learner Outcome 


Year 2 
J*lean 


Year 3 
Mean 


t 


df 


P 


1.1 


Finding and gathering 
information and ideas 


1.79 


1.88 


-2.37 


2658 


.02 


1.2 


Organizing 

and manipulating 

information and ideas 


1.77 


1 . 85 


-2 . 10 


2661 


. 04 


1.3 


Expressing information, 
ideas and emotions 


1.67 


1.77 


-2.74 


2661 


< . 01 


2.0 


Conceptual understanding 


1.65 


1 . 84 


-5 . 16 


1915 


< . 01 


3.1 


Task commitment 


1.91 


2 . 00 


-2.41 


2685 


.02 

t 


3.2 


Self-concept : 
self understanding 


1.32 


1.14 


3.78 


2047 


< .01 


3.3 


Self-concept: humor 


0.89 


0.76 


2 . 71 


2440 


< .01 


4.1 


Cooperation 


1.43 


1.41 


0.23 


1925 


. 82 


4.2 


Broadened perspectives : 
rights and responsibilities 


1.14 


1.19 


-1.05 


1874 


.29 


4.3 


Broadened perspectives : 
different points of view 


1.06 


1.18 


-2 . 56 


1819 


.02 


5.1 


Strategies 
in problem solving 


1.51 


1.63 


-3.08 


2640 


< .01 


5.2 


Reasoning 

in problem solving 


1.43 


1.52 


-2 . 32 


2659 


' .02 


5.3 


Creativity 
in problem solving 


1.43 


1.41 


0.53 


2158 


.59 


6.1 


Integration of ideas 


1.19 


1.46 


-6 . 14 


2604 


< .01 


6.2 


Connection of life and 
classroom 


1.18 


1.45 


-6 . 05 


2596 


< . 01 


6.3 


Learning how to learn 


1.54 


1.70 


-4.46 


2646 


< .01 
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Table 3 

Mean Scores and Independent £-Tests Comparing Year 2 and Year 3 



Subjects for Performance 


Tasks Rated 


on Six 


Learner 


Goals 




Kentucky Learnina Goal 


Year 2 
Mean 


Year 3 

Mean _t 


df 


£ 


1. Applying basic skills 


1 . 74 


1.83 


-2.58 


2665 


.01 


2 . Concepts 


1.64 


1 . 84 


-5 . 16 


1915 


< .01 


3. Self-sufficiency 


1.38 


1.30 


2.08 


2511 


< . 04 


4 . Team member 


1 . 21 


1.27 


-1.29 


1882 


.21 


5 . Problem solving 


1.46 


1 . 52 


-1.69 


2663 


.09 


6 . Integration 


1.30 


1 . 54 


-6 . 18 


2598 


< .01 



Note . Means for learning goals shown above were obtained by averaging 



over learner outcomes (see Table 2) . Goals and learner outcomes were as 



follows : 

Goal 

1 

2 

3 

4 

5 

6 



Learner outcomes averaged 

1.1 1.2 1.3 

2.0 

3.1 3.2 3.3 

4.1 4.2 4.3 

5.1 -5.2 5.3 

6.1 6.2 6.3 
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Data on Portfolios Table 4 shows mean ratings on complete portfolios for Year 2 and Year 
3 subjects, and it also shows results of independent t-tests used to compare the means. As with data 
in Tables 2 and 3, the t statistic was reported with either standard or downward-adjusted degrees of 
freedom, depending on the outcome of an equality of variances test. 

The analysis revealed that, using the significance criterion of p < .05, there were significant 
differences in favor of the Year 3 group for 15 of the 16 assessment criteria. On one criterion, number 
3.3, there was no significant difference between the groups. In addition to the univariate tests 
reported above, a multivariate analysis of variance (MANOVA) was performed, with the 16 portfolio 
ratings as dependent variables and the treatment vs. control designation as the independent variable. 
The n for the MANOVA consisted of 2471 cases. The multivariate Pillai's trace criterion was . 04483, t 
which converted to a statistically significant F(16, 2454) = 7.20, p < .0005. The multivariate test 
corroborated the general superiority of the Year 3 data, found in the results of the separate t-tests. 

In addition to the data on the 16 separate ratings for the portfolios, additional data were 
analyzed. New variables were created that were averages over subsets of the 16 rating scale 
categories; the separate ratings within each goal were averaged to yield six goal rating items. Table 
5 shows means and t-test comparisons for the six goal areas. All goal areas were statistically 
significant in favor of the Year 3 group. A MANOVA on the six average goal areas yielded a 
multivariate Pillai's trace criterion of .02414 which converted to a statistically significant 
F(6, 2618)= 10.79, p< .0005. 
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Table 4 

Mean Scores and Independent £.-Tests Comparing Year 2 and Year 3 
Subjects for Portfolios Rated on 16 Learner Outcomes 



Year 2 Year 3 



Goal 


Learner Outcome 


Mean 


Mean 


_ t 


df 


P 


1.1 


Finding quid gathering 
information and ideas 


1.69 


1.85 


-4 . 18 


1677 


. < .01 


1.2 


Organizing 

and manipulating 

information and ideas 


1 . 75 


1 . 86 


-3.08 


2770 


< .01 


1.3 


Expressing information, 
ideas and emotions 


1.73 


1 . 84 


-3 . 05 


2769 


< .01 


2.0 


Conceptual understanding 


1.63 


1 . 92 


-7 . 18 


1605 


< .01 


3.1 


Task commitment 


1.93 


2 . 11 


-4.60 


2766 


*c .01 


3.2 


Self-concept: 
self understanding 


1 . 60 


1 . 70 


-2.20 


2726 


. 03 


3.3 


Self-concept: humor 


1 . 15 


1 . 19 


-0.98 


1694 


.33 


4'. 1 


Cooperation 


1.62 


1 . 73 


-2.22 


1701 


.03 


4.2 


Broadened perspectives : 
rights and responsibilities 


1.43 


1, 53 


-2.13 


2654 


.03 


.4-3 


Broadened perspectives : 
different points of view 


1.33 


1.43 


-2 . 02 


2661 


.04 


5.1 


Strategies 
in problem solving 


1 . 54 


1.73 


-4 . 91 


1648 


< .01 


5.2 


Reasoning 

in problem solving 


1 . 53 


1.66 


-3.31 


2761 


< .01 


5.3 


Creativity 
in problem solving 


1.54 


1 . 72 


-4.63 


2760 


< .01 


6.1 


Integration of ideas 


1 . 28 


1 . 53 


-5 . 75 


2728 


< . 01 


6.2 


Connection of life and 
classroom 


1.24 


1.53 


-6.57 


2731 


< .01 


6.3 

ERIC 


Learning how to learn 


1.49 

52 6 4 


1 . 57 


-2.08 


1790 


. 04 
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Table 5 

Mean Scores and Independent t-Tests Comparing Year 2 and Year 3 
Subjects for Portfolios Rated on Six Learner Goals 



Kentucky Learnincr Goal 


Year 2 
Mean 


Year 3 
Mean 


t 


df 




n 


1 . . Applying, basic skills 


1.72 


1.84 


-3.67,.. 


2773 


< 


. 01 


2 . Concepts 


1.63 


1.91 


-7.18 


1605 


< 


.01 


3. Self-sufficiency 


1.57 


1.67 


-3.12 


2735 


< 


.01 


4 . Team member 


1.46 


1.56 


-2.29 


1654 




. 02 


5 . Problem solving 


1.54 


1.70 


-4.71 


2769 


< 


.01 


6 . Integration 


1.34 


1.54 


-5.29 


2738 


< 


. 01 



Note . Means for learning goals shown above were obtained by averaging 



over learner outcomes (see Table 4) . Goals and learner outcomes were as 
follows : 



Goal 

1 

2 

3 

4 

5 

6 



Learner outcomes averaged 

1.1 1.2 1.3 

2.0 

3.1 3.2 3.3 

4.1 4.2 4.3 

5.1 5.2 5.3 

6.1 6.2 6.3 
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Effects of additional variables on ratines 

Several analyses were undertaken to examine the effects of background variables on ratings 
of performance tasks and portfolios. These were exploratory and should be followed up with 
additional tests in the future. They will be explained only briefly. 

Only , data from Year 3 were analyzed. In a previous report, the author has already discussed 
analyses for Year 2 data. The dependent variables were: (a) the six performance task averages 
(similar to the data shown in Table 3), and (b) the six portfolio averages (similar to the data shown 
in Table 5). 

Ethnicity The student sample was overwhelmingly White Non-Hispanic in ethnicity. 
However, there were 94 African-American children, and it was decided to compare their ratings with 1 
those of Whites. On the performance task averages, independent t-test comparisons favored whites 
(p < .05) on goal areas 1, 2, and 5. However, multivariate analysis of variance (MANOVA) did not 
reveal significant ethnic group differences for the set of sue performance task ratings. On the portfolio 
averages, independent t-test comparisons favored whites (p < .05) on goal areas 1, 2, 3, and 5. The 
multivariate analysis of variance (MANOVA) also showed a significant effect favoring whites. 

Gender and economic level A factorial MANOVA was performed using the six performance 
task averages as the dependent variables and two independent variables: gender and economic level. 
On the performance task ratings, using .05 as the type I error probability criterion, there was no 
significant interaction effect, a main effect of gender, and a main effect of economic level. Separate 
univariate factorial analyses of variance were performed for each of the sue dependent variables, and 
the same pattern existed for each of these: no interaction effect and two main effects. Regarding the 
main effect of gender, females had higher scores than males. Regarding the main effect of economic 
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level, scores went up according to the level of the student. In other words, mean scores were lowest 
for students on free lunch, mean scores were somewhat higher for students on reduced lunch, and 
mean scores were highest for students who paid for lunch. 

When a factorial MANQVA was performed using the six portfolio averages as the dependent , 
variables and gender and economic level as the independent variables, similar results occurred. There 
was no interaction effect, females has higher scores than males, and more affluent children had higher 
scores than less affluent children. 

More research is warranted on how economic level and the other background variables affect 
teacher ratings of student performance. Such analyses could include discussion of the following* 
issues: magnitude of effects (i.e., calculation of effect sizes and power analysis), possible biases or 
other problems due to small n in certain cells, and the effects of missing data on results. 
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